Skip to content

Add support to mine npm PackageURLs#726

Merged
AyanSinhaMahapatra merged 17 commits into
mainfrom
minecode-pipeline-npm
Dec 10, 2025
Merged

Add support to mine npm PackageURLs#726
AyanSinhaMahapatra merged 17 commits into
mainfrom
minecode-pipeline-npm

Conversation

@AyanSinhaMahapatra

@AyanSinhaMahapatra AyanSinhaMahapatra commented Sep 29, 2025

Copy link
Copy Markdown
Member

@AyanSinhaMahapatra AyanSinhaMahapatra force-pushed the minecode-pipeline-npm branch 2 times, most recently from cd445c6 to c210044 Compare September 29, 2025 08:54
Reference: #661
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Reference: #664
Reference: #660
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
* Fix npm packages JSON file structure
* Compress package names JSON to reduce size and allow git push

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@JonoYang

Copy link
Copy Markdown
Member

@AyanSinhaMahapatra I've merged in #728 for the dependency updates, so you'll need to deal with the merge conflict there

Comment thread minecode_pipelines/notebooks/pypi.json Outdated
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra

Copy link
Copy Markdown
Member Author

@JonoYang I've resolved the conflicts.

Also npm packageURLs are being mined allright at:

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Comment thread minecode_pipelines/miners/npm.py Outdated
@keshav-space

Copy link
Copy Markdown
Member

Let's use MineCodeBasePipeline to make sure these mined purls are deployed in git repos.

@pombredanne

Copy link
Copy Markdown
Member

Pimg... anything left todo here?

Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
Reference: #798
Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
@AyanSinhaMahapatra

Copy link
Copy Markdown
Member Author

@keshav-space keshav-space left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @AyanSinhaMahapatra LGTM, just few stray debug prints.

Comment thread minecode_pipelines/miners/npm.py Outdated
Comment thread minecode_pipelines/miners/npm.py Outdated
Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
@AyanSinhaMahapatra AyanSinhaMahapatra merged commit 785907d into main Dec 10, 2025
1 of 2 checks passed
@AyanSinhaMahapatra AyanSinhaMahapatra deleted the minecode-pipeline-npm branch December 10, 2025 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants